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ON THE COHOMOLOGICAL EQUATION FOR NILFLOWS 


LIVIO FLAMINIO AND GIOVANNI FORNI 

Abstract. Let AT be a vector field on a compact connected manifold 
M. An important question in dynamical systems is to know when a func¬ 
tion 5 : M —> R is a coboundary for the flow generated by X, i.e. when 
there exists a function f : M ^ M. such that Xf = g.ln this article we 
investigate this question for nilfiows on nilmanifolds. We show that there 
exists countably many independent Schwartz distributions such that 
any sufficiently smooth function 5 is a coboundary iff it belongs to the 
kernel of all the distributions 


1. Introduction 

1.1. The problem. For a detailed diseussion of the eohomologieal equa¬ 
tion for flows and tranformations in ergodie theory, we refer the reader to 
IIKat03l . Here we limit ourselves to a brief, self-eontained introduetion. 

Let A be a smooth veetor field on a eonneeted eompaet manifold A4 
and let (0x)teK be the flow generated by A on M. Many problems in 
dynamies and ergodie theory (see loe. eit.) can be reconducted to the study 
of the eohomologieal equation 

(1) Xu = f, 

i.e. the problem of finding a function u on Ad whose Lie derivative Am in 
the direction of A is a given function f on AA. Clearly a continuous solution 
u of the equation ([T]) is only determined up to function constant along the 
orbits. In addition, given the value of u at one point p E AA, the solution 
u is uniquely determined on the whole orbit of p under the flow (0x)teK- 
In fact, the recurrence properties of flow well forbid the 

existence of measurable solutions to O. Hence the subtlety of the problem 
lies entirely in the fact that, given / in a certain regularity class, we may or 
may not have solutions u in some other regularity class. 

For example, observe that if the eohomologieal equation O admits a 
continuous solution u then 

(2) /i(/) = 0 for all A-invariant Borel measures p E C*{A4). 
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Thus invariant measures are obstructions to the existence of solutions of o 
in the continuous class. Livshitz’s celebrated theorem states that if X is 
an Anosov vector field and / is Holder continuos of exponent a G (0,1), 
then the necessary condition Q is also sufficient and that, in this case, the 
solution u is actually Holder continuos of exponent a. (This theorem has 
been generalized to other hyperbolic dyanamical systems and to smoother 
classes of functions, see for example |KH95| ). 

For uniquely ergodic flows, i.e. for flows admitting a unique invariant 
measure, one may wonder whether the above condition @ is sufficient. It 
is well known that, for X a constant vector field on a torus, 

• if X is Diophantine, for any function / of average zero there 
exists a C°° solution u of equation dH); 

• if X is Liouvillean, then there exists some function / of average 
zero there for which there is no measurable weak^ solution u of 
equation dU). 

In general, all invariant distributions (in the sense of Schwartz), which 
are not necessarily signed measures, are obstructions to the existence of 
smooth solutions of the cohomological equation. 

A distribution D G is called X-invariant if XD = 0 in the dis¬ 

tributional sense, that is, if D{X(j)) = 0 for all (j) G C^{Ai). Let X“(X1) 
be the space of all X-invariant distributions in V'lAA). 

By definition, if u G C“(X1) is a solution of dB, we must have B(f) = 0 
for all D G Z^(A4). By the same token, if m is a solution of dB which 
is (a + l)-times differentiable, then we must have D(f) = 0 for all X- 
invariant distributions D G V\Xi) of order a > 0. 

Let W°‘{M) be the Sobolev space of square-integrable functions with 
square-integrable (weak) derivatives up to order a > 0 and let T^(A4) C 
X^(Xl) be the space of X-invariant distributions of Sobolev order a > 0, 
i.e. which extend continuously to the Sobolev space IF"(XI). 

If M G is a solution of dB> then D{f) = 0 for all D G X^(A4). 

It is then natural to ask if and when invariant distributions are a complete 
set of obstructions to the existence of differentiable solutions, i.e. when the 
condition 

(3) / G IL"(X1) and D(f) = 0 for all B G Ix(M) , 

is sufficient to guarantee the existence of a solution u satisfying some regu¬ 
larity properties (for example u G for some k G M"*"). 

The main results of this paper is that this is the case for all Diophantine 
nilflows on compact nilmanifolds. 


fihat is [u, Xfj)) = — (/, (j)) for all (j) G C°°{M) 
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1.2. Main result. Let n be a fc-step nilpotent real Lie algebra (k > 2) with 
a minimal set of generators £ := {Ei, ..., En} C n. Let Uj, j = 1, ..., fc, 
denote the descending central series of n: 

(4) ui = n, ns = [n, n],..., n],..., n^ C Z(n), 

where Z (n) is the center of n. 

Let N be the connected and simply connected nilpotent Lie group with 
Lie algebra n. The corresponding Lie subgroups Nj = exp N] 

form the descending central series of N. 

Let r be a lattice in N. It exists if and only if N admits rational structure 
constants (see, for example, | |Rag72|iCG90ll l. 

A (compact) nilmanifold is a by definition a quotient manifold J\f : = 
r\A^ with N a nilpotent Lie group and L C iV a lattice. 

On a nilmanifold J\f = T\N, the group N acts on the right transitively 
by right multiplication. By definition, the nilflow (0x)igR generated by 
A G n is the flow obtained by restriction of this action to the one-parameter 
subgroups (expfA)ieR of N: 

(5) 0|y(rT) = rxexp(fX). 

It is plain that nilflows on T\N preserve the probability measure on T\N 
given locally by the Haar measure. To simplify the notation, the vector field 
on T\N generating the flow {(j)x)teR will also be indicated by X. 

Every nilmanifold is a fiber bundle over a torus. In fact, the group N = 
A/[A, A] is abelian, connected and simply connected, hence isomorphic 
to M" and T = r/[r, T] is a lattice in A. Thus we have a natural projection 

(6) p : r\A ^ r\A 

over a torus of dimension n. 

We recall the following: 

Theorem 1.1 HIGrebll . HAGHbSI f. The following properties are equivalent. 

(1) The nilflow ((0^)ieR, p) on V\N is ergodic. 

(2) The nilflow (0^)igR on V\N is uniquely ergodic. 

(3) The nilflow (0^)igR on V\N is minimal. 

(4) The projected flow ('^^)iGR on V\N ^ T" is an irrational linear 
flow on hence it is (uniquely) ergodic and minimal. 

The “irrationality” condition above in Theorem 11.11 (4), refers to the 
rational structure determined by the lattice T. Namely, if the generators 
[El, ..., En} of n are chosen so that the elements {exp Ai,..., exp 
project onto generators (exp Ai,..., exp A„} of T, then there exists a vec¬ 
tor fix := (a;i(A),... ,a;n(A)) G M"' such that 

(7) X = a;i(A)Ai -f ■ ■ • -h a;„(A)A„ 
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and the condition means that ... ,ujn{,X) are linearly independent 

over Q. We shall call such an element X G n irrational (with respect toT). 
We have: 

Theorem 1.2. Let M = T\N be any compact nilmanifold obtained as a 
quotient of a k-step nilpotent (k > 2) connected, simply connected Lie 
group N by a lattice V d N. For any irrational X G n, the space of X- 
invariant distributions (A/*) has countable dimension. In fact, (M) 
admits a countable basis Ex of invariant distributions of Sobolev order 
1/2, in the sense that Ex C X^ (A/”) for any a > 1/2. 

We shall say that X G n is Diophantine (with respect toV) of expo¬ 
nent r > 0 if the projection X of X in n/[n, n] is Diophantine of exponent 
r > 0 for the lattice f in the standard sense; namely, if there exists a con¬ 
stant X > 0 such that, for all M := (mi,..., m„) G Z" \ {0}, 

(8) |(M,flx)| := |^m,a;,(X)| > • 

The subset of Diophantine elements X G n of exponent r > 0 will be 
denoted by DCT-(n). The subset DC(n) := U.rDCT-(n) of all Diophantine 
elements has full Lebesgue measure in the Lie algebra n. 

Let 1 Tq"(A/’, X) := VL“(A/’)nkerX^(A/') for any a > 1/2. Wethenhave: 

Theorem 1.3. Let J\f, T, N be as above. If X E DCT-(n) the following 
holds. 

(1) If a > n-\-T — 1/2 and P < —1/2, there exists a linear bounded op¬ 
erator Gx '■ fL"(A/”) — W^(N') such that, for all f G the 

distribution u = Gxf is a weak solution of cohomological equa¬ 
tion (UJ). 

(2) If a > n + T and P < [a — (n + T)][(n + r)fc + 1]“^, there ex¬ 
ists a linear bounded operator Gx '■ Wf(N', X) —> W^(N'), such 
that, for all f G W°'(M), the function u = Gxf is a solution of 
cohomological equation Q. 

1.3. Motivations and applications. We have already mentioned that the 
cohomogical problem for a flow (or a transformation) enters in many prob¬ 
lems in dynamical systems (KAM, time-changes, etc.). We recall the fol¬ 
lowing definitions: 

Definition 1.4. Let M. be a compact connected manifold. A smooth vector 
field X on AA is 

(a) globally hypoelliptic (GH), if Xu G G°°{M) implies u G G°°(A4) 
for any distribution u G V'(M); 
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(6) cohomology free (CF), if for all f G C^{M) there exists a constant 
c{f) G C and u G such that 

Xu = f - c{f). 

In IIGW73I Greenfield and Wallaeh showed that, if a veetor field X on 
Ai is globally hypoelliptie, then there exists an invariant volume form uj 
and the spaee of X-invariant distributions is redueed to the line Cu (in 
partieular the X-flow is uniquely ergodie). The same is true, by a simple 
exereise, if X is eohomology free. In faet, if X is eohomology free, then it 
is globally hypoelliptie. 

Question 1. Is a globally hypoelliptie vector field on a compact connected 
manifold also cohomology free? 

Definition 1.5 ( IIKat0IllKat03ll ). A vector field X on a compact connected 
manifold M. is said G°°-stable if the subspace {Xu \ u G is 

closed in 

It is plain that for G^-stable veetor fields the answer to the above ques¬ 
tion is positive. Absenee of stability is related to fast approximation by pe- 
riodie systems: the major example of C'^-wnstable are Liouvillean eonstant 
veetor fields on tori (loe.eit.). In IIGW73I , Greenfield and Wallaeh showed 
that if a Killing veetor field of a Riemannian metrie on Ai is globally hy¬ 
poelliptie, then the manifold Ad is a torus and that on the flat 2-torus any 
Killing globally hypoelliptie veetor field must be a eonstant Diophantine 
veetor field. Motivated by these results they eonjeetured: 

Conjecture 1 (' IIGW73II '). If a compact, connected manifold Ai admits a 
globally hypoelliptie vector field X then Ai is a torus and X is smoothly 
conjugate to a constant Diophantine vector field. 

In faet, if Ad is a torus, by BCCOOII any globally hypoelliptie veetor field 
X is smoothly eonjugate to a eonstant Diophantine veetor field. 

A related eonjeeture due to A. Katok is the following: 

Conjecture 2 ( IIKatO 1 1 IHur85 1 iKatUH ). If a compact, connected manifold 
Ad admits a cohomology free vector field X then Ai is a torus and X is 
smoothly conjugate to a constant Diophantine vector field. 

In support of Katok’s conjecture Federico and Jana Rodriguez Hertz have 
recently proved in IRHRHQSI that any (compact, connected) manifold Ad 
admitting a cohomology free vector field fibres over a torus of dimension 
equal to the first Betti number of Ad. 

Theorems o and 11.31 immediately imply that, within the class of nil- 
flows, both conjectures are true. As remarked by the Rodriguez Hertz’s in 
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fRHRHOSl . nilflows are the simplest elass of flows for whieh their theorem 
yields no non-trivial information. 

Another motivation for the study of eohomologieal equations and invari¬ 
ant distributions eomes from “renormalization dynamies” (see IIFor02al i. In 
faet, whenever we ean define, for a family of “parabolie flows”, an effeetive 
renormalization dynamies on a suitable moduli spaee, the analysis of the 
aetion of the renormalization on the bundle of invariant distributions over 
the moduli spaee may allow to determine the asymptoties of the ergodie av¬ 
erages for the typieal flow in the given family. This program was earned out 
in llFor02bll for eonservative flows on (higher genus) surfaees (related to in¬ 
terval exehange transformations), in iFF03bl for horoeyele flows on hyper- 
bolie surfaees and in HFFOSal for nilflows on the 3-dimensional Heisenberg 
group. These results have shown that the phenomenon of polynomial devia¬ 
tions of ergodie averages for interval exehange transformations, diseovered 
by A. Zorieh IIZor94L llZor96l . IIZor97l and rigorously proved in IIFor02bL 
is shared by other fundamental examples of parabolie flows. In prineiple, it 
should be possible to earry out this program by purely dynamieal methods 
based on renormalization. In faet, in the ease of interval exehange transfor¬ 
mations, Marmi, Moussa and Yoeeoz IMMYOSI have been able to replaee 
the harmonie analysis methods applied by the seeond author in his study of 
the eohomologieal equation IIFor97ll by a renormalization approaeh (based 
on the Rauzy-Veeeh-Zorieh induetion). However, no renormalization dy¬ 
namies is eurrently available for general nilflows. 

2. Irreducible Unitary Representations 

2.1. Kirillov’s Theory. Let N be the eonneeted and simply eonneeted nil- 
potent Lie group with Lie algebra n. By Kirillov theory, all the irredueible 
unitary representation of N are parametrized by the coadjoint orbits O C 
n*, i.e. by the orbits of the coadjoint action of N on n* defined by 

(9) Ad*(^)A = AoAd(^-^) geN^Xen*. 

For A G n*, the skew-symmetrie bilinear form 

(10) Bx{X,Y) = X{[X,Y]) 

has a radical X\ whieh eoineides with the Lie subalgebra of the subgroup 
of N stabilizing A; thus the form (Col) is non-degenerate on the tangent spaee 
to the orbit (9 C n* of A and defines a sympleetie form on O. 

A polarizing (or maximal subordinate) subalgebra for A is a maximal 
isotropie subspaee m C n for the form Bx whieh is also a subalgebra of n. 
In partieular any polarizing subalgebra for the linear form A eontains the 
radieal Xx- If m is a polarizing subalgebra for the linear form A, the map 

expT I—>■ exp27r?A(T), T G m, 
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yields a one-dimensional representation, which we denote by exp of 
the subgroup M = exp m C iV. 

To a pair A := (A, m) formed by a linear form A G n* and a polarizing 
subalgebra m for A, we associate the unitary representation 

(11) tta = Ind^p„(exp27r«A). 

These unitary representations are irreducible; up to unitary equivalence, all 
unitary irreducible representations of N are obtained in this way. Further¬ 
more, two pairs A := (A,m) and A' ;= (A',m') yield unitarily equivalent 
representations tta and tta' if and only if A and A' belong to the same coad- 
joint orbit C> C n*. 

The unitary equivalence class of the representations of the group N de¬ 
termined by the coadjoint orbit O will be denoted by lio, while we set 

Ha = {tta I a = (A, m), with m polarizing subalgebra for A}. 

Definition 2.1. Let N be a connected, simply connected Lie group of Lie 
algebra n. The derived representation tt* of a unitary representation tt ofN 
on a Hilbert space is the Lie algebra representation ofn on defined 
as follows. For every X G n, 

(12) 7r*(X) := strong-lirn(7r(exptX) —/)/t. 

It can be proved that the derived representation tt* of the Lie algebra n 
on Hj, is essentially skew-adoint in following sense. For all X G n, the 
linear operators 7r*(X) are essentially skew-adjoint with common invariant 
core the subspace of c of C°°-vectors in We recall that 

a vector v G is for the representation vr if the function ge N ^ 
7r{g)v G FFtt is of class as a function on N (with values in a Hilbert 
space). 

Suppose that N is the semi-direct product A ix X' of a normal subgroup 
N' and an abelian group A and that tt' is a unitary irreducible representation 
of N' on an Hilbert space iF'; then the derived representation tt* of the 
induced representation tt = Indj^/ {tt') has a simple description in terms of 
7r(: in fact, up to unitary equivalence, ^ L‘^{A,H'); furthermore the 
subgroup A acts by translations, and hence for / G L‘^{A, H') we have 

(13) (7r*(X)/) (a) = ^ /(aexptX)I^^Q , a G A, X G a ;= Lie(A); 

the infinitesimal action of N' is a pointwise action given explicitely by: 

(14) (7r,(X)/) (a) = < (Ad(a)y) (/(a)), a G A, X G n' ;= Lie(X0. 

Since, for any a G A, the operator Ad(a) acting on n' is unipotent, the 
right-hand side in the formula above is a polynomial function in the variable 
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a G A. In fact, if i is the degree of nilpoteney of ad(X) and a = exp X, we 
have, for all F G n' = Lie(iV'), 

(15) MY)f) (o) = (/(“)) ■ 

j=o 

2.2. Coadjoint orbits of maximal rank. Let n be a fc-step nilpotent real 
Lie algebra on n generators Ei,... ,En. Let xij, j = 1,... ,k, denote the 
descending central series of n: 

m = n, 02 = [n, n],..., n^ = n],..., C Z(n). 

In this seetion we eharaeterize eoadjoint orbits that eorrespond to unitary 
representations whieh do not faetor through the quotient iV/expUfe. Sueh 
eoadjoint orbits and the indueed unitary representations will be ealled of 
maximal rank. Sinee the Lie group iV/expnfc is {k — l)-step nilpotent, the 
analysis of unitary representations of maximal rank is suffieient to treat by 
induetion all unitary representations of a given fc-step nilpotent Lie group. 

We shall make the fundamental assumption that the eoadjoint orbit O C 
n* has maximal rank (see below). Before stating this assumption, a few 
lemmas and definitions. 

Let 

nti(A) = {.Y e n I ni_,) = 0 }, 

nGi(O) = nGi(A), for any A e O . 

Sinee the restriction of A G n* to the eentre Z{n) does not depend on 
the ehoiee of the linear form A G (9 and sinee [n, n^.i] = C F(n), 
the restriction x and the subspaee n^_^(A) depend only on the 
eoadjoint orbit O. Consequently, the seeond definition in (fT^ is well-posed. 

Lemma 2.2. Let O be a eoadjoint orbit and A G (9. Then 

(1) tA C 

(2) na C 

(3) is a sub-algebra. 

Proof. Condition (1) is immediate from the definitions. Conditions (2) 
and (3) follow from Jaeobi’s identity and the inelusion C F(n). In 
faet, if 5A(Ti,nfc_i) = Bx{T 2 ,nk-i) = {0} then 5 a([Ti,T s],n^-i) = 
A([[Ti,T 2 ],Ufc-i]) C —A([[T 2 , Ufc-i],Ti]) — A([[nfc_i,Ti],T 2 ]) = {0}, sinee 
[T,-,nfc_i] c Ufc C F(n). 

□ 

Lemma 2.3. Let O be a coadjoint orbit and A G (9. The following proper¬ 
ties are equivalent: 
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(1) the restriction A|nfc is identically zero 

(2) n^_i(C>) = n 

(3) the projection ofn^_^{0) on n/n 2 is surjective 

Proof. (1^2) Since [n, n^-i] = n^, if the restrietion A|nfc is identieally 
zero, by definition, we have n^_^(A) = n. 

(2^1) From [n, Wk-i] = n^, if n^_i(A) = n, we have A|nfc = 0. 

( 3<^2 ) Any subalgebra of n eontaining n 2 and whieh projeets onto n/n 2 
must eoneide with the full algebra n. □ 

If the eonditions of the above lemma are satisfied, any vr G Tio faetors 
through a representation of N/ exp Ufc, a{k — l)-step nilpotent group. Hav¬ 
ing in mind an induction process on the degree of nilpoteney k, we shall 
assume that we are not in this ease. 

Definition 2.4. If none of the conditions of the Lemma IQ are satisfied 
we will say that the coadjoint orbit O, any linear form X G O and any 
irreducible representation vr G Hq have maximal rank ( equal to k). 

Thus from now on we shall foeus on eoadjoint orbits O of maximal rank. 


2.3. Adapted representations. Let O be any eoadjoint orbit of maximal 
rank. In Seotion|3we shall study the eohomologieal equation for a “generie” 
admissible X G n restrieted to an irredueible unitary representation vr G 
Yio- There we shall see, in faet, that the set of admissible X G n equals 

n\ni_fO). 

The following lemma produees, for any coadjoint orbit O of maximal 
rank and any X G n \ n^_^(C>), a special irreducible unitary representation 
TT G Ho adapted to our goal of proving a priori estimates for the eohomo¬ 
logieal equation restrieted to that representation. 


Lemma 2.5. Let X G n \ x\.^_i{0) and let Y G n^-i be any element such 
that Bx{X,Y) f II for all X E O. There exists a codimension 1 ideal 
n' C n with X ^ n' and a unitary (irreducible) representation tt G Hq with 
the following properties: 

(1) the representation tt is obtained inducing from N' := exp x\! to N a 
unitary irreducible representation tt' of N' on a Hilbert space H'; 

(2) the derived the representation vr* of the Lie algebra n satisfies 

(a) 7r*(X) = §on L^(R, H', dt), 

(b) 7r*(F) = 2TnBx{X,Y)t\dH' on ,dt); 

(3) there exists a constant C > 0 such that 


\{X,U)\>C-^ 


Bx{XX)\ 


(17) 
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where U & n denote the unit normal vector to n' with respect to an eu¬ 
clidean product (•, •) on n fixed once for all. 

Proof. Since, by Lemma we have 1 x 2 C for any X e O, the 

subspaee n' = {T G n | B\{T,Y) = 0} is a eodimension 1 ideal of n 
depending only on the eoadjoint orbit O. Let U G n be a normal unit vector 
to n' with respect to an euclidean produet (■, ■) on n fixed onee for all. There 
is an orthogonal decomposition 

(18) X = {X,U)U + W, 

for some fL G n'. Sinee by definition of n' we have Bx{W,Y) = 0, it 
follows immediately that 

(19) Bx{X,Y) = {X,U)Bx{U,Y). 

There exists a eonstant C > 0 sueh that ||f/|| = ||F|| = 1 implies that 
il [U, 1^] II <C and Y G xik-i implies [U, Y] G n^. Hence 

(20) |i?A(f/,r)| = |A([(7,F])|<C||A|n,||. 

The lower bound (El) follows immediately from (O and (|20l). 

Let m C n' be a polarizing subalgebra for the restrietion A' = A|n' 
of A to n'. Then m is also a polarizing subalgebra for A on n ( IICG90L 
Prop. 1.3.4), since, by definition, we have X\ C n' for all X e O. 

Set 

A = (A, m), A' = (A', m), vr = tta and vr' = tta'. 

By induetion in stages (see for example IICG90I . §2.1), we have: 

TT = Ind^p^(exp(zA)) Ind^p„, (^Ind“P;;;(exp(*A'))) = Ind^p„,(7r'). 

From (Eii-dni), if we identify with R the one-parameter subgroup generated 
by X and if we denote by H' the Hilbert spaee on with tta' aets, up to a 
unitary equivalenee, the representation tta coincides with the representation 
of N on L^(R, H\ dt) infinitesimally given by: 

7i^{X)f{t) = f/(f), 7r*(F)/(f) = <(Ad(expfX)F)/(f), 

for F G n' and all / in a dense subspace of L^(R, if', df), sueh as the 
subspace G“(R, if') of compaetly supported C°° funetions from R to if'. 
Since [X, F] G n^ C Z{n) we have [X, [X, F]] = 0 and therefore 

(21) Ad(expfX)F = F + f[X,F]; 

we also have C n', henee C F(n') C m' and 7r'|nfc = expzA; from 
(fT5b . we obtain that vr*(F) is the operator pointwise given by: 

7r,(F) = f<([X,F]) +<(F) = 27r*ffi,(X,F)IdH' + <{Y). 
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We claim that there exist X G O and a representation n' G IIa' such 
that vr^F) = 0. This will prove the lemma. 

Since [X, Y] G Z{n), the value Bx{X, Y) does not depend on A G C>. 
Let A G O. From (l2Tt we have 

(Ad*(exptoX)A)(F) = A(Ad(exp(-toX))y) = X{Y) - t,B,^{X,Y) 

which vanishes for to = /B\{X, Y). Thus we can and shall assume 

that we have chosen a linear form A G (9 satisfying A(y) = 0. 

Let n^ = {T G Ufc I A(T) = 0}. By hypothesis n'^ is a central ideal in n 
of codimension 1 in Ufc. We set 

h' = 

and we denote by A' the linear form induced on h^ by A'. We also denote by 
p' : N' = exp n' ^ N'/ exp 

the canonical projection. We have IIa' = {vr' o p' | tt' G 11^^/} since the 
central ideal is contained in every subalgebra m C n' polarizing for A' 
and every representation vr' G IIa' is trivial on expn^'. But now observe 
that the projection Y of F in h' belongs to the centre F(h') of h'. In fact, 
if T G n', we have [T, F] G n^ and then the condition B\{T,Y) = 0 is 
equivalent to [T, F] = 0 mod n'^. We conclude that for every Y G we 
have 7f'(F) = *A'(F) = 0 and consequently vr'(F) = 0 for any tt' G Ha'. 
This concludes the proof. □ 

3. The cohomological equation in representation 

3.1. Distributions and Sobolev spaces. Let X be a connected, simply 
connected Lie group. Let tt be a unitary representation of X on a Hilbert 
space Ht,. Let vr* denote the derived representation of the Lie algebra n 
of X on Ht,. The subspace c is endowed with the Frechet 

C°° topology, that is, the topology defined by the family of seminorms 
{ II ■ II Ei,E2,...,Em I m G N and Ei, ..., Em G n} defined as follows: 

II V II EuE2,...,e^ ■= II vr*(Xi)7r*(L;2) ■ ■ ■ 7r4Em)v || , for i; G . 

Definition 3.1. Let e be a unitary representation of the Lie algebra n on 
the Hilbert space Ht,. The space of Schwartz distributions for the repre¬ 
sentation TT is defined as the the dual space ofC°°{HT^) (endowed with the 
Frechet C°° topology). The space of Schwartz distributions for the repre¬ 
sentation vr will be denoted 

The representation vr* extends in a canonical way to a representation 
(denoted by the same symbol) of the enveloping algebra U{n) of n on 
the Hilbert space X^. Let A G X(n) a left-invariant second-order, pos¬ 
itive elliptic operator on X fixed once for all. For example, the operator 
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A = —{V^^ -)> where {Vi,..., Vd} is a basis of n as a veetor spaee. 

For all a G M+, let C be the Sobolev spaee of veetors in the 

maximal domain of the essentially self-adjoint operator (J -f 7r*(A))“^^ en¬ 
dowed with the Hilbert space norm 

||f||a,:= II (/-f 7r*(A))“^^i; II , for all f G hF“(-ff7r) • 

Let also c be the dual Hilbert space of Then 

is the projective limit of the spaces (and consequently 

is the inductive limit of W~°‘{Ht,)) as a — -|-cx). A distribution 
D G will be called a distribution of order at most a G M’*'. 

Definition 3.2. Let n be a unitary representation of the Lie group N (with 
Lie algebra n) on the Hilbert space H-„. The cohomological equation for 
X E nin the representation vr is the linear equation 

(22) 7r*(A)M = / (for the unknown vector u G 

where f G is a given (sufficiently smooth) vector. 

Definition 3.3. For any X G n, the space o/X-invariant distributions/or 
the representation n is defined as the space Tx(Ht,) of all distributional 
solutions D G VfHf) of the equation n^(X)D = 0. Let 

If(H^) :=Ix{H^)nW-^(H^) 

be the subspace of invariant distributions of order at most a G M"*". 

It is immediate that all X-invariant distributions in VfHfij yield obstruc¬ 
tions to the existence of solutions u G C°°(H^) of the cohomological 
equation and that X-invariant distributions of order at most a G M’*' yield 
obstructions to the existence of solutions u G 

Let C> be a codjoint orbit of maximal rank and let vr G Tio be an induced 
irreducible (adapted) unitary representation of N on = L^(M, H') con¬ 
structed as in Lemma 12.51 The space of of (7°° vectors for the 

representation tt can be characterized as follows. 

Let S (R, C°°(H')) be the space of Schwartz functions on R with val¬ 
ues in the Frechet space of (7“ vectors for the representation vr' on H' 
(cf. Lemma IT51) . The space S (R, C°°(H')) is by definition endowed with 
the Frechet topology induced by the family of seminorms 

{|| ■ ||i,i,Ei,i? 2 ,...,E^ |z,/m G Hand G n'} 

defined as follows: for all / G 5 (R, C^(H')), 

(23) \\f\fj,EuE 2 ,...,E^ ■■= sup 11(1 + f^)^/^vr'(Ei) ■ ■■'K((E^)f^\t)\\H' . 
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Lemma 3.4. As topological vector spaces, 

Proof. Let X, F G n and n' C n be as in Lemma |2.51 By formula (1151) . for 
any E en' and for all f e S (R, C°°{H')), 

(24) ME)f) (i) = ^ (ad(X)^B) /(i). 

We claim that the Schwartz space S (R, C°°{H')) C and that the 

embedding is continuous. In fact, for any / G 5 (R, C°°{H')), we have 

(25) ll/ll < sup 11(1+ , 

teM 

hence it follows from formula (El that there exists a constant Ck > 0 such 
that for any E e n' and all / G 5 (R, 


(26) 


\T^*{E)f\\ < Ck max 

0<j<k 


sup + (ad(X)^E) /(f)II//' 

teR 


In order to simplify the notation, let E^ ■= ad^Xy E G n' for any £' G n' 
and for any j G {0,..., A;}. By estimate (l26b and by iterated applications 
of the identity (El, it follows that for any £ G N there exists a constant 
Ck,i > 0 such that, for any Ei,..., G n' and for all / G 5 (R, 


(27) Wiry El) ■■-Try Ei)f\\ < 


< Ckf max 

’ o<n,...,ji<k 


sup 11(1+ i^)5 ]/('"< (£■«■>)/Wll«' 

a=l 


hence, for any Ei,..., Ei e n', any z G M and any / G 5 (R, C°°{H')), 


(28) ||7r,(Ei)---7r,(E/)7r,(X)7|| < 

sup 11(1+f2)in /'“’WIIh- . 

teR 1 

Q:=l 

Since [X, n'] C n' the claim follows. 

Conversely, we claim that C°°{Hy c 5 (R, C°°{H')). In fact, let / G 
C^{Hy be a vector. Since 7r*(X) is the operator / i—/', we have 
/ G (7°°(R, H'). Since 7r*(y) equals, up to a non-zero (scalar) factor, the 
pointwise operator f{t) i—>• tf{t), we have {t i—>■ Pf{t)) G L^(R, 77'), for 
all j > 0, hence / G 5(R, 77'). Thus 

(29) C'“(77^) C 5(R,77') 


< Ck,e max 
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and we therefore have a eontinuous linear operator 

For any E E x\! and any j G N, let (g) 'k'^{E) be the densely defined 
linear operator on L^(R, El') uniquely determined by the identities 

£t[T^ ® n',{E)]f := EK{E)£tf , for alH G M. 

We will prove by (finite) induetion that, for all a G {0,..., fc — 1}, if 
E G Ufc.Q, n n', then we have, for all j G N and for all / G C°°{H^), 

/ 6 dom[TJ0 <(£)], 

' [A 0 <(£)]/ e . 

Remark that, sinee [T^ ® n'^{E)] equals 7r*(F)'^ [l ® Tii{E)] up to a non¬ 
zero sealar faetor. the seeond line of formula (EDI) aetually follows from 

(31) [l®7r:(E)]/GC°"(Ff.). 

If a = 0, for any Eq G C Z{x\) the operators tt^{Eq) and are 

sealar multiples of the identity. Henee the elaim (|30|) holds in this ease. 

Let us assume for all E G x\k-a and for all j G M. Let Ea+i £ 
nfc_(Q,+i) n n'. Remark that the elements 

El^^ := ad(X)^E„+i G nfc_„ n n', for all /? G {1,..., a + 1} . 

By formula we have the identity, 

a+l -| 

(32) = 5,7r.(£)„+i)/ - 0 . 

13=1 

where 7r*(E„+i)/ G and [T* ® vr^i^i^^)]/ G C°°{H^) by the 

induetion hypothesis, sinee / G It follows by the inelusion (l2^ 

and the identity (1^ that the elaim (BUt holds for all E^+i G Wk-a H n' and 
all j G N. The induetion step is therefore eompleted. 

It follows from (l30l) by another induetion argument (on f G N), that, for 
all E G n' and all i, j G N, we have 

/«Gdom[T^®<(E)], 

[T^'®<(E)]/« GC“(i7j. 

In faet, for i = 0 the eonditions cs reduee to Col). Assume that C3 holds 
for z G N and for all j G M. Sinee [1 0 7 r((L^)]/W g C°°{H^), we have 

[1 ® = 7r,(X)[l (8) 7r:(i?)]/« G C 5(M, H'), 
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hence /h+i) ^ dom[l (g) Next, for j e N \ {0}, we have the 

following immediate identity: 

(34) [T^®7r:(E)]/(*+') = 7r,(X)[T^®7r:(E)]/«-j[T^'-'(8)7r:(E)]/«, 

which allows to conclude that (l3^ holds for z + 1. The induction argument 
is therefore completed. 

Finally, by successive applications of (El) we have that, for any i, j G N, 
any Ei, Em G n' and for all / G 

|r> ■<(£„)!/« 6c“(i/,). 

In conclusion, we have proved that S (M, C°°{H')) C and that 

the inclusion is continuous and surjective, hence it is an isomorphism of 
Frechet spaces by the open mapping theorem (or by a direct argument based 
on the proof of surjectivity given above). □ 


3.2. Main Estimates. In this section we describe all invariant distributions 
and we prove Sobolev bounds for the Green operator of the cohomological 
equation in each irreducible unitary representation of maximal rank (deter¬ 
mined by a coadjoint orbit O of maximal rank) for any admissible vector 

Xen\ni_,{0). 

Let (■, ■) be an euclidean product on the Lie algebra n fixed once for all 
and let II ■ || denote the corresponding norm. 

Let O be any coadjoint orbit. Since the restrictions A|nfc and i?A|n x 
Ufc-i do not depend on the choice of the linear form A G (9, the following 
definitions are well-posed. 

Let 


(36) 


Wk{0) : = ||A|nfc|| = 


wz{0) : = ||A|Z(n)|| 


|A(^)| 

max ; 

.^GnfcVfO} ll-^ll 

|A(^)| 

= max ——— 

xex(n)\{0} ||Z|| 


Since C Z{n) we have Wk{0) < wz{0). 
For all (X, X) G n x nfc_i, let 


(5o(X,F) := \Bx{X,Y)\ , for any A G O, and 
5o(X) := max{(5o(X, F) | Y G and ||y|| = 1} . 

By definition we have 6oiX) > 0 if and only if X ^ The latter 

condition is non-empty if and only if O has maximal rank, in which case it 
holds except for a subspace of positive codimension. 
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Lemma 3.5. Let 6o{X) > 0, and vr G lio irreducible essentially skew- 
adjoint representation o/n on a Hilbert space Let Xx{Ht,) C 
be the space of X-invariant distributions, i.e. of linear functionals which 
vanish on Ran {X\C°°{Ht,)) C The space has a count¬ 

able basis Bx{Htt) with the following properties: 

(1) each D G Bx{H-k) has Sobolev order equal to 1/2, in the sense that 

D G for any a > 1/2; 

(2) for each D G Bx{Htt) tind each a > 1/2, there exists a constant 

C ■.= C{D, a) > D such that the following holds: for any Y G Ufc-i 
with 6o{X, F) > 0 and for all f G we have 


(38) \D{f)\<C6o{X,Yr^/^\\{I-7i.{Y)^r/^f\\ . 


Proof Let C> c n* be a coadjoint orbit such that So{X) > 0, let Y G rik-i 
be such that 6o{X,Y) > 0 and let vr G Hq- Since the statement of the 
theorem depends only on the unitary equivalence class of the representa¬ 
tion, we can assume by Lemma IT5] that vr is obtained from an irreducible 
representation vr' of an ideal n' C n of codimension 1 on a Hilbert space 
H' by adjoining X G n \ n', hence vr*(X) is the derivative operator ^ on 
H/ dt), and that vr^F) = 2mt5o{X, Y) Id^'- 
Let B' c Lf' be a basis of the Hilbert space H' and let Bx{Ht/) be the set 
of linear functionals D^, e G B', defined as follows. For any e G B', for all 
/ G 



(39) 


By Holder inequality, for all a > 1/2 there exists a constant > 0 such 
that the following (a priori) estimate holds: 



hence G for all a > 1/2; in particular G 'D'(iF^). 

Clearly Dg is invariant by translations, hence De G Ix{H.„). It remains 

to be proved only that Bx{Ht,) is a basis of Xx{Hn)- Since by Lemma IT?I 

we have = S if De{f) = 0 for all e G B', then 

/jg/(s)(is = 0 G and the function 



It follows that D{f) = 0 for all D G Xx{Ht/), since there exists g G 
such that / = 'K{X)g. Hence, by the Hahn-Banach theorem, 
Bx{Ht/) is a (countable) system of generators for the (closed) subspace 
Xx{Ht/) C It is immediate to verify that, since B' C H' is linearly 
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independent, the system is also linearly independent, henee it is a 


basis 


□ 


Theorem 3.6. Let 6o{X) > 0, and let tt G Ho be an irreducible essentially 
skew-adjoint representation of n on a Hilbert space 

(1) There exists a Green operator Gx '■ defined 

for all a > 1/2 and (3 < —1/2, such that, for all f G W°‘{Hf), the 
distribution u := Gxf is a solution of the cohomological equation 
'K,,{X)u = /; there exists a constant G := G{X, a) > 0 such that 


\\Gxfh<G5o{X)-^\\f\\ 


(2) If f G a > 1, and D{f) = D, for all D G Tx^Ht,), then 

Gxf e Wf^{H^),forall p < {a — 1) /k; furthermore there exists a 
constant G' := C(X, a, /?) > 0 such that 

\\Gxfh < G' max{l,w;fc(C>)^} max{l, ||/||„ . 

Proof Let (9 C n* be a eoadjoint orbit and let tt G Hq. If 5c> := ^o(^) > 
0, by Lemma IZ5] we ean assume that the representation vr is obtained from 
an irredueible representation vr' of a eodimension 1 ideal n' on a Hilbert 
spaee H' by adjoining X G n \ n', henee 7r*(X) is the derivative operator ^ 
on Ht, = L^(M, Hj df), and that there exists Y G nfc_i sueh that 7r*(y) = 
2t:i 16a Id///. In faet, by eompaetness of the unit sphere in nfc_i, there exists 
Y G Ufc-i sueh that ||y || = 1 and So{X, Y) = 6o > 0. 

The operator Gx ■ ^" 0 ( 1 ^, H') B{R, H’) defined for / G Co(M, H') by 
(41) Gxf{t)'.= [ f{s)ds, foralHGM, 


— 00 


admits a bounded extension Gx ■ ^ Gb(M, H'), for all a > 1/2; 

in faet , if / G a > 1/2, by Holder inequality there exists a 

eonstant > 0 sueh that 



(42) 


A similar estimate shows that ( 7 ^( 1 ^, H') C W for all a > 1/2. In 

faet, if M G Cb(M, H'), for all a > 1/2 there exists Cq, > 0 sueh that, if 
u G 


( 43 ) 
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It follows that Gx '■ is well-defined and bounded for 

all a > 1/2 and all /9 < —1/2, and there exists Ca ,/9 > 0 such that, for all 

/ e 

(44) WGxSh < ll/llo 

Oo 

By construction 7r^{X)Gxf = / in for all / G 

a > 1/2 and (3 < —1/2, and Gxt^*{X)u = u m for all u G 

a > 1/2 and /3 < -1/2. 

Finally, if / G ll^"(iF^), for a > 1/2, and D{f) = 0, for all D G 

mH.), 

/ t p-\-oo 

f{s)ds = — f{s)ds, foralltGM. 

OO J t 

It follows that 


r*+oo 


^+oo 


(46) ||7r,(F)^G'x/|r < / {\27iSot\‘^ \f{s)\wds] dt 


Since, for all o; > 1, 


-f / {\27r6otf / \f{s)\H'ds) dt . 


(47) 


'^al 



-fCX) , 


by Holder inequality and change of variables, 

(48) IMYYGxfW < ^ II (/ - n^Yf) f\\ . 

Oo 

Let il(n) and il(n') be the enveloping algebras of n and n' respectively. 
Clearly from the inclusion n' C n we have that it(n') is contained in it(n) 
as a subalgebra. In addition, since n' is an ideal of n, (Ad(exptX))teR 
is a one-parameter group of automorphisms of n' which extends to a one- 
parameter group (expt(5)teR of automorphisms of it(n'). The generator 5 
of this group of automorphisms is the derivation on it(n') obtained by ex¬ 
tending the derivation ad(X) of n' from n' to il(n'). Observe that from the 
nilpotency of n it follows that, for any P G if(n') there exists a first integer 
[P] such that (jIP+^P = 0. 
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We recall that by Lemma IT4l we have = iS(R, C°°{H')). Let 

8t : be the linear (evaluation) operator defined by 

= f{t)- By definition of the representation vr* on (cf. for¬ 

mula (fTSt f. we have, for any P G if(n') and any E G n', 

[^1 p 

(49) 8t 7r,(P) = Ti^ie^^P) 8t = Y, all f G M , 

i=o P 

from which we obtain, for all s, f G M, 

(50) = 8t 7r,{e-^^P) = • 

j=o P 

Since the action of 7r*(L) on C°°{H^) can be rewritten as 

(51) 8t7r^{Y^) = {2'Ki5oty 8t , for all j eN and f G M , 
setting f = s in the (l5Ut we obtain, for all f G M, 

[^1 (-l)i 

( 52 ) x;(F) e, = £Me-"P) = 5 ; ' ., g. ir.iY’Slp ). 

Since 8tGx = 8s ds we have, for all f G M, 


8MP)Gx = <{P^P)8,Gx = <(e“P) / ds 

J —oo 

[p p pt [p pj pt 

Y^-p.(.s=P) £.* = ^-7/ 

„_n J—oo •_n J‘ J—oo 


ds 


3=0 

P (-1)^ G 

^K‘^m6oY j-c 


3=0 


£,7r*(LV+^P) ds 


[p [p p ( 

^8tGx7rYY^6^+^P) 


[P] [^] 




j=o e=o 
[P] 

«.E 


m=0 ^ l+j=m 
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hence, using (l48t . for all a > 1, there exists a constant Ca\p] > 0 such that 
for all / e we have 


(53) ||7r,(P)Gx/|| < 
IP] 




E - 


1 c 




j + a 


—^ (27r5c))™' ^ 

m=0 ^ l+j=m ^ 

[^’1 


(I-T(Yf) ‘ 7r.{Y‘S’’'P)f\\< 




m=0 


Let A the positive definite Laplacian associated to an euclidean product 
(•, ■) on the Lie algebra n. Then for any orthonormal basis P of n we have 
A = — YjVgb ^ G n be a normal unit vector to n'. Without loss of 

generality we can assume that U e B. We set Aq = — Y^vgbmu} 
have Aq G il(n') and A = —(7^ + Aq. Clearly [Aq] < 2{k — 1) and [Ag] < 
2(fc — l)r. It follows bv (15^ that there exists a constant C := C{k,r,a) > 0 
such that 


(54) ||7r*(A;;)G'x/|| < C max{l,5(p^ 

2(fc-l)r 

m=0 

Since (7 is a unit vector orthogonal to n', there exists IL G n' such that 

(55) U = {X,U)~\X -W). 

Since n' C n is an ideal and 7i{X)Gxf = f, it follows by estimate (flTt in 
Lemma IT31 that there exists a constant C" := C'{k, r, a, A) > 0 such that 

||ir,(t;)"''Gx/|| < (||,r.(A - WpGx/W 

(56) ^ ^ ,, 

< c (Il/I|2r-1 + ||ir.(A„)’'G;f/|| ) , 

By bounds (l54l) and (l5^ . there exists C := C''{k, r, a^X) >0 such that 

(57) WGxfhr < G" max{l, Wk{Of'^} max{l, ()o^“^''^}||/|| 2 fcr+a • 

Finally by interpolation for all a > /3 > 0 such that a > 1 + /c/3 there exists 
a constant Ga,^ ■= Ga^pik, A) > 0 such that 

(58) \\Gxfh < Ca,f3 max{l, Wk{Of} max{l, ||/||„ . 

□ 
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4. The cohomological equation for Nilflows 

4. 1. The Howe-Richardson multiplicity formula. Let be a /c-step nil- 
potent Lie group with a minimal set of generators {Ei,..., En} C ni. Let L 
be a lattice in N. The nilmanifold M = V\N carries a unique A^-invariant 
probability measure fi, locally given by the Haar measure of A^; for each 
X G n we denote by (0x)teM the the flow on M given by the right action of 
the one-parametre subgroup {expfX | f G M}. 

The Hilbert space jj) decomposes under the right action of N into 

a countable Hilbert sum Hi of irreducible closed subspaces Hi. 

The Howe-Richardson multiplicity formula BHowTll IRic71l tells us 
which irreducible representations from the unitary dual N of N appear 
in the decomposition Hi. To state the formula, recall that a couple 
(x, M) is called a maximal character if there is A G n* and a polarizing 
subalgebra m C n for A such that: 

(1) M = expm; 

(2) X is the one-dimensional representation of M defined by 

expVL G M ^ e 2 «A(w). 

a maximal character {x, M) is called maximal for T if, in addition, the 
following conditions are satisfied: 

(3) M intersects T into a lattice, i.e. M fl T\M is compact; 

(4) X is trivial on M fl T: fl T) = {!}. 

As we recalled in El when (x, M) ranges over the set of maximal charac¬ 
ters, the family of representations IndM(x) exaust the unitary dual N. The 
multiplicity formula states that IndM(x) appears in /r) iff (x, M) is 

a maximal integral character for T, with multiplicity given by the cardi¬ 
nal of closed orbits xM of M on A/” such that x(StabM(T)) = {!}• This 
discussion motivates the following: 

Definition 4.1. A linear form A G n* is called integral (with respect 
to T) if there exists a polarizing subalgebra m C n such that the pair 
(exp(27rzA), exp m) is a maximal integral character for T. A coadjoint orbit 
O C n will be called integral (with respect to T) if some X e O is integral 
(hence all X E O are). 

Definition 4.2. Let T be a lattice in N. A linear form A G n* is called 
weakly integral (with respect to T) if X{E) G Zfor all E G log Z{T). A 
coadjoint orbit O C n will be called weakly integral (with respect to T) if 
some X E O is weakly integral (hence all X E O are). 

Since Z(r) = rnZ(A^) C exp m for any maximal polarizing subalgebra 
men, any integral linear form (coadjoint orbit) is also weakly integral. 
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In fact, since the pair (exp(27r*A), expm) is a maximal integral eharacter, 
exp {2mX{W)) = 1, henee A(iy) e Z, for all W E logZ(r). 

It follows from the above diseussion that all irredueible unitary sub¬ 
representations occurring in fi) correspond are induced by Kirillov’s 

by integral eoadjoint orbits. 

4.2. Diophantine elements. Let A the abelian group of rank n given by 
A = N/{N,N) and let f = r/(r,r); then f is a lattiee in A. Let 
{exp El,, exp E^} denote a set of generators of L, with Ei E Lie(y4) Ri 
n/[n, n]. It is plain that the EiS form a basis of n/[n, n]. 

We shall say that an element Af G n is Diophantine (with respect toV) 
of exponent r > 0 if the projection X of X in n/[n, n] is Diophantine of 
exponent r > 0 for the lattiee f in the standard sense: that is, if we let 
X = ijJi{X)Ei + ... uJn{X)En, there exists a eonstant iL > 0 sueh that, for 
all M = (mi,..., nin) E'IA\ {0}, 

K 

(59) |(Af,Jix)| ■ 

The set of Diophantine elements X G n of exponent r > 0 will be denoted 
byDCT-(n). The subset of all Diophantine elements DC(n) := UT-DCT-(n) C 
n has full measure. 

Let Ef Ef ...E^^,Ef,...,El^,...,E'[,..., E^^, (with rii = n), a 

Malcev basis for n through the descending eentral series Uj and strongly 
based at T, that is a basis of n satisfying the following properties: 

(1) if we drop the first i elements of the basis we obtain a basis if a 
subalgebra of eodimension i of n. 

(2) if we set := {E{, ..., E^.j the elements of the set U U 
■ ■ ■ U form a basis of 

(3) every element of T ean be written as a produet 

exp m\El ... exp ... exp niiEi ... exp 

with integral coefficients m^. 

The existenee of sueh a basis is obtained eombining the proof of Theo¬ 
rems 1.1.13 and 5.1.6 of IICG90I . 

It is also elear that we ean assume that the basis [Ei ,..., En} of n/[n, n], 
appearing above, is given by the projeetions in n/[n, x(\oi E\,El,... Ef 
We define an Euelidean produet on n by making the basis E^ orthonor¬ 
mal. The norm of n will be the one indueed by this produet. 

Lemma 4.3. Let X E DCr{n). Let O be a eoadjoint orbit of maximal rank 
weakly integral (with respect toT) and let vr G Ilo. There exists a constant 
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Cr > 0 such that for all f G W"' we have: 

(60) 6 oiX)-^f\\<Cr\\f\\n-i+r. 

Proof. Let A G (9. Since O is weakly integral (with respect to F) and has 
maximal rank, we claim that there exist Y G such that 

(61) My := Y),..., B,{El Y)) G Z" \ {0} . 

In fact, if F G then [Ej, Y] G n^, since C nfc_i. By the Baker- 
Campbell-Hausdorff formula, we have (exp exp y) =exp[i?j,y]. It 
follows that [Ej,Y] G logF(r) and, since A is weakly integral, we have 
Bx{Ej,Y) = X{[Ej,Y])eZ for all j G {1,..., n}. If 5 a(£:], F) = 0, for 
all F G and all j G {1,..., n}, then = n (see Lemma 

contradicting the hypothesis that A has maximal rank. The claim is proved. 
Since F G by definition we have ||F|| = 1. Hence 

(62) So(X) > |Ba(A',F)| > |{My,(ix)| > . 

Let TT G Tfo be an irreducible representation of the Lie algebra n on the 
Hilbert space H := Let P = — (27r)“^ F]^ G il(n). Since 

[Ei,Y] G Z{n), for all i G {!,... ,n}, the operator 7r*(P) = |MypIdj)/. 
Hence for any / G iF we have 

(63) 6o(A')-‘||/|| < A'-‘|||My|"-‘+Vll = A'-'||7r(P)T/|| , 

Since P has order 2 as an element of the enveloping algebra U (n) (in fact 
P < (27r)“^A), if / G IF"“^+'^(FF), the inequality ||7r*(P)" /|| < 

||/|U_ 1 +, holds. □ 

4.3. Uniform estimates. We prove that, for any Diophantine element, ap¬ 
propriate Sobolev norms of the Green operators for the cohomological equa¬ 
tion, which have been constructed in each irreducible unitary represen¬ 
tation, are bounded uniformly over all irreducible unitary representations. 
This step is crucial in order to solve the cohomological equation on a nil- 
manifold J\f = T\N by ’’gluing” together the solutions constructed in every 
irreducible sub-representation of the representation of N on jj). 

Lemma 4.4. Let O be any coadjoint orbit and let tt G Tfo be a unitary 
representation of N . For all f G the following inequality holds: 

(64) «>z(0)ll/ll<T ll/ll,. 

Proof By the definition OEl of the non-negative number wz{0), for each 
coadjoint orbit O of N, there exists an element Z G F(n) such that ||Z|| = 
1, with respect to the fixed euclidean norm on n, and |A(Z)| = wz{0), for 
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any A G C>. In addition, since Z G Z{x\), for any vr G lio we have that 
7r(Z) = It follows that, for any / G 

(65) ,„^(o)||/|| = ±||,r.(z)/||<± 11/11,, 

as elaimed. □ 

Corollary 4.5. Let X G DCr(n) . Let O be a non-trivial coadjoint orbit, 
integral with respect to a lattice F, and let vr G Tlo cin irreducible represen¬ 
tation of the Lie algebra non a Hilbert space H.,^. 

(1) If a > n + r —1/2 and f3 < —1/2, there exists Cf := Cf (X, a) > 0 
such that, for all f G 

\\Gxfh<C^\\f\\a-, 

(2) if f G a > n + r, and D{f) = 0, for all D G 

then Gxf G for all jd < [a — {n-\- r)][(n + T)k + 1]“^ 

and there exists C/ := G'^^X, o;, /?) > 0 such that 

\\Gxfh<GW\U. 

Proof The proof is by induetion on the degree of nilpoteney k G N\{0}of 
the nilpotent group X. If fc = 1 the group N is abelian, all irredueible uni¬ 
tary representations are one-dimensional and (weakly) integral. The Dio- 
phantine eondition immediately implies that the statement holds in this ease 
with the exeeption of the trivial representation. Let us assume that the state¬ 
ment holds for all lattiees in any (k — l)-step nilpotent group and let N be 
a fc-step nilpotent group. If the eoadjoint orbit O is of maximal rank and 
integral with respeet to a lattiee F c X, then the statement follows from 
Theorem 13.61 Lemma Inl and Lemma l4^ 

If O is integral (with respeet to F), but it is not of maximal rank, then for 
any A G C>, by Lemma the restrietion A|nfc is identieally zero and any 
irredueible representation vr G Hq faetors through an irredueible represen¬ 
tation vr' of the (A; — l)-step nilpotent group X' := X/ expUfc, indueed by 
the linear funetional A' G (n/n^)*. Sinee A G n* is integral (with respeet 
to F), there exists a polarizing subalgebra m sueh that (exp(27r?A), exp m) 
is a maximal integral eharaeter for F. Sinee A|nfc = 0, the subalgebra 
m' := m/ufc is polarizing for A' and (exp(27rzA'),expm') is a maximal 
integral eharaeter for the lattiee F' := F/F fl exp n^. By the induetion hy¬ 
pothesis, the statement holds also in this ease. □ 

4.4. Proof of Theorems 11.21 and I1.3L Let L‘^{J\f, p) = be the de- 

eomposition of the spaee of square integrable funetions on M into irre¬ 
dueible eomponents under the right aetion of X. Sueh a deeomposition 
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induces a decomposition of the subspace of smooth vectors, 

C-(Ar) = 0C'“(i7O, 

iGN 

and of the Sobolev spaces: for each a e M, there is an orthogonal splitting 

( 66 ) = 

iGN 

Consequently, for any X e n, the spaces XxiM) C of all X- 

invariant distributions and the subspace (Af) := Ix (Af) fl W~°'{Af) split 
as follows: 


Ix^AT) — 0 Jx(-ffj); 

(67) 

JGN 

Let X be irrational. By Lemma 13.51 and by the decompositions (IFTb . the 
space Xx{Af) admits a countable basis Bx{Af) C XJ, for any a > 1/2. 
This proves Theorem 1 1.21 

Let X G DCT-(n). Then the Green operator for the cohomological equa¬ 
tion on A/” can be constructed as follows. Leto; > n+r— 1/2 and< —1/2. 
For any i G N, let be the Green operator for the cohomological equa¬ 
tion for X in the irreducible representation Hi. By Corollary 14.51 the oper¬ 
ators : W°‘{Hi) are bounded and have uniformly bounded 

norms, in the sense that there exists Gf > 0 such that ||G^^ < Gf. 

For each i G N and a G M, let p^a : W°‘{Af) W^i^Hi) be the orthog¬ 
onal projection and ja^ : W°‘{Hi) —> W°‘{Af) the embedding, determined 
by the orthogonal splitting (|^. The operator Gx : W°‘{Af) —> W^{Af) 
defined as 

Gx :=04'’»G!‘'opB 

iGN 

is a well-defined, bounded Green operator for the cohomological equation 
for the Diophantine vector field X G DCT-(n) on Af. In fact, by the orthog¬ 
onality of the splitting (1^ . 


||G 


X\\a,l3 < 


iGN 


\\jf 


O Gx ° 


°Po 


i,i3 ^ ill/3 ^ (6^f)" 


Finally, if o; > n + t and / G W°‘{Af) belongs to the kernel of all X- 
invariant distributions D G X^(A/’), then u := Gx{f) e W^{Af) for any 
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{3 < [a — {n + r)][(n + T)k + 1] By construction, 

“ = 5^4*^ (PaH/)) • 

ieN 

By the splitting (1^ . it follows that Di{pa\f)) = 0, for all Di G XJ(-ffj) 
and for each i G M, since D{f) = 0 for all D G X^(A/’). Hence, by 
Corollary 14.51 there exists a eonstant > 0 sueh that, for all i G N, 

I|gJ'(p®(/)) llp<cf||p®(/)||„. 

Again by the orthogonality of the splitting (1^ . we have 

Ml = Y. M'-S (p®(/)) lip s (G?)"E s (G?)"II/IIJ ■ 

ieN jGN 

The proof Theorem [O] is concluded. 
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